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Abstract —Backscattering is a sensitive probe of the accuracy 
of electron scattering algorithms Implemented in Monte Carlo 
codes. The capability of the Geant4 toolkit to describe realistically 
the fraction of electrons backscattered from a target volume 
is extensively and quantitatively evaluated in comparison with 
experimental data retrieved from the literature. The validation 
test covers the energy range between approximately 100 eV and 
20 MeV, and concerns a wide set of target elements. Multiple and 
single electron scattering models implemented in Geant4, as well 
as preassembled selections of physics models distributed within 
Geant4, are analyzed with statistical methods. The evaluations 
concern Geant4 versions from 9.1 to 10.1. Significant evolutions 
are observed over the range of Geant4 versions, not always in 
the direction of better compatibility with experiment. Goodness- 
of-fit tests complemented by categorical analysis tests identify 
a configuration based on Geant4 Urban mnltiple scattering 
model in Geant4 9.1 version and a configuration based on single 
Coulomb scattering in Geant4 10.0 as the physics options best 
reproducing experimental data above a few tens of keV. At lower 
energies only single scattering demonstrates some capability to 
reproduce data down to a few keV. Recommended preassembled 
physics configurations appear incapable of describing electron 
backscattering compatible with experiment. With the snpport 
of statistical methods, a correlation is established between the 
validation of Geant4-based simulation of backscattering and of 
energy deposition. 

Index Terms —Monte Carlo, simulation, Geant4, electrons 


I. Introduction 

T he simulation of backscattering is a sensitive playground 
to evaluate the capability of a Monte Carlo transport code 
to describe electron multiple scattering accurately. Multiple 
scattering modeling affects not only directly associated ob¬ 
servables, such as the fraction of electrons impinging on a 
target that are backscattered, their energy spectrum and angular 
distribution, but also the simulated energy deposition in the 
target volume. 

Quantitative evaluations Q.Q of the capability of Geant4 
0 ^ 0 to reproduce high precision measurements of the 
energy deposited by low energy electrons in various targets 
0^@ hint at a signihcant contribution of multiple scattering 
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implementations to determine the accuracy of the simulated 
energy deposition. 

The study documented in this paper analyzes quantitatively 
the simulation of electron backscattering based on Geant4. 
It evaluates the performance of several Geant4 physics con- 
hguration options, in an extended series of Geant4 versions, 
with respect to a large sample of experimental data collected 
from the literature, which cover the energy range from 100 eV 
to 22 MeV approximately. Compatibility with experiment is 
established by means of goodness-of-ht statistical methods, 
while the different ability of Geant4 physics modeling options 
to reproduce experimental data is quantihed by the statistical 
analysis of categorical data derived from the outcome of 
goodness-of-ht tests. Finally, the results of this validation 
process are correlated with the outcome of the validation of 
electron energy deposition in Q, and the signihcance of this 
correlation is quantihed. 

The scope of the paper is limited to testing the electron 
backscattering fraction simulated by Geant4. Apart from the 
considerable amount of material needed to document this 
subject alone, the results reported in section VI suggest that 
validation tests of more complex observables, such as the 
spectrum and angular distribution of backscattered electrons, 
would be more meaningful once the Geant4 multiple scattering 
domain has benehted from the opportunities for improvement 
highlighted in this paper for future versions of the toolkit. 

The results of this validation test provide guidance to 
optimize the selection and conhguration of electron scattering 
models in experimental application scenarios based on several 
Geant4 versions examined in this paper. They also provide 
a quantitative ground for future improvement of the physics 
modeling and the software development process in Geant4 
electromagnetic physics domain. 


II. Electron backscattering 

Electron backscattering has been the subject of experimental 
and theoretical interest for several decades. A review on this 
topic is outside the scope of this paper; the brief overview 
in this section has the purpose of summarizing information 
relevant to the simulation validation test documented in this 
paper. 


A. Experimental data 

A large number of experiments have measured various 
features related to electron backscattering: the fraction of 
backscattered particles from targets of various thickness and 
material composition, the angular distribution of backscattered 
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electrons and their energy spectrum, surface effects on solid 
targets and other effects associated with material properties. 
These experimental observables provide information to inves¬ 
tigate the underlying physical effects. 

Typical experiment arrangements involve an electron beam 
of defined energy, high purity targets and a detector to measure 
the electron current in the backward hemisphere with respect 
to the target. A hemispherical grid is often used to discriminate 
electrons with energy above a preset threshold, with the intent 
of reducing the contamination from secondary electrons in 
the detected electron sample. The fraction of backscattered 
electrons is determined as the ratio between the number 
of electrons reaching the detector and the total number of 
primary electrons. Experimental uncertainties are associated 
with the electron beam energy, beam current and current 
leakage. The detected electron sample can be contaminated 
by particles scattered from components of the experimental 
apparatus other than the target, or by secondary electrons. 
Detailed descriptions of measurement setups can be found in 
0 and 0. 

The present study collected more than 3000 experimental 
data 0- | [6^ from the literature, concerning the measurement 
of the electron backscattering fraction from infinite or semi¬ 
infinite targets (i.e. of size exceeding the electron range in the 
target material, or half of its value). The experimental sample 
encompasses 48 target elements, which span the range of 
atomic numbers from beryllium to uranium. Electron energies 
vary from 78 eV to 22.2 MeV. The analysis reported in the 
following sections concerns measurements performed with a 
normally incident beam. 

Apparent inconsistencies are visible in the experimental 
data sample, regarding measurements performed by different 
experimental groups in similar nominal configurations of target 
composition and primary electron energy: they hint at the 
presence of systematic effects. Several references do not report 
any experimental uncertainties, nor evaluate possible sources 
of systematic errors affecting the measurements. In some cases 
experiments performed with similar techniques report largely 
different uncertainties, which hint to possibly underestimated 
errors. 

The experimental data are shown in the figures of this paper 
with error bars corresponding to the uncertainties reported 
in the related references. The apparent absence of error bars 
associated with some data points reflects either the omission of 
experimental uncertainties in their published source or values 
smaller than the size of the data point markers in the plots. 

B. Simulation with general purpose Monte Carlo codes 

The simulation of electron backscattering has been a subject 
of interest for general purpose Monte Carlo codes, such as 
EGS @, llig, ELUKA @, Geant4 0, Q, ITS 
MCNP | |68t and Penelope | |69) , as it is a sensitive instrument to 
demonstrate the capabilities of their electron transport models, 
namely the treatment of multiple and single electron scattering. 

The models implemented in these codes are based on a rela¬ 
tively limited set of theoretical approaches fTO) , developed by 
Goudsmit and Saunderson | |7T1 , | |72l , Moliere^^, Lewis lzl)> 


and Wentzel eg. complemented by algorithms specifically 
tailored to the application of the theory to the Monte Carlo 
particle transport environment. Some of these algorithms are 
documented in the literature (e.g. eg), other are only 
described in the software documentation of the Monte Carlo 
systems; nevertheless, limited information is usually available 
about their theoretical grounds, methods of approximation and 
implementation details, and the evolution of the software is 
often the source of inconsistencies between the description of 
the algorithms in the literature or the software documentation 
and their actual implementation. 

Comparisons of electron backscattering simulations with 
experimental data are reported in the literature: some exam¬ 
ples are H) and concerning EGS5 |[64) and EGSnrc, 
respectively, | |M| , concerning MCNP4B |8l| and EGS4 | [82) , 
[Sg , concerning ELUKA. The comparisons available in the 
literature are limited to a single source or a small collection 
of experimental data: this limitation exposes them to the risk 
of biased conclusions, if the reference data are affected by 
unidentified systematic effects, or are not adequately repre¬ 
sentative of the variety of scenarios encountered in particle 
transport applications regarding electron energies and target 
composition. They rest on a qualitative appraisal of the com¬ 
patibility between simulation and experiment. 

To the best of our knowledge, this paper is the first 
quantitative report, based on rigorous statistical methods, of 
the comparison of electron backscattering simulation based 
on a major Monte Carlo system with respect to an extensive 
collection of experimental data. 

III. Electron Multiple Scattering Simulation in 
Geant4 

The electromagnetic physics package of Geant4 encom¬ 
passes processes and models, which deal with electron sin¬ 
gle and multiple scattering with the interacting medium. In 
addition, the physies lists package of Geant4 collects a set 
of classes, which instantiate predefined selections of physics 
processes and models for several particle types: they can be 
directly employed in simulation applications by users not 
willing to develop physics selections specific to their own 
experimental scenario. 

A. Processes and models 

Geant4 deals with the scattering of electrons with matter 
by means of two processes, multiple and single Coulomb 
scattering, which are specializations of the continuous-discrete 
and discrete processes handled by Geant4 tracking algorithm, 
respectively. Part of the functionality associated with these pro¬ 
cesses is delegated to multiple or single scattering models. A 
process can be configured with one or more associated models, 
which implement different algorithms and may cooperate to 
describe electron scattering over different energy ranges. The 
UML (Unified Modeling Language) class diagram in Eig. [T] 
illustrates the main features of multiple and single electron 
scattering in Geant4 10.0 and 10.1. 
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Fig. 1. UML (Unified Modeling Language) class diagram illustrating Geant4 multiple and single scattering processes (dark grey) and models (light grey) 
involved in this study, and their relationship with Geant4 abstract bases classes (in italic) interacting with the tracking kernel. The diagram is pertinent to 
Geant4 10.0 and 10.1 versions. 


The Urban multiple scattering mode l ||84) -|[86), in¬ 
tended to be based on Lewis’ theory |74|, is instanti¬ 
ated by default in Geant4 electron multiple scattering pro¬ 
cess. Several variants of this model have been released 
in the course of the evolution of Geant4; G4MscModel, 
G4UrbanMscModel, G4MscModel71, G4UrbanMscModel90, 
G4UrbanMscModel2, G4UrbanMscModel92, G4UrbanMsc- 
Model93, G4UrbanMscModel95 and G4UrbanMscModel96. 
Table U lists the default model associated with the Geant4 
versions considered in this paper. Some relevant parameters 
characterizing the multiple scattering algorithm are described 

izl- 

Simulation of single electron scattering is implemented 
in Geant4 G4CoulombScattering process and G4Coulomb- 
ScatteringModel model | [88| . This model is based on Wentzel 
calculations CD; it has been available in Geant4 since the 9.0 


TABLE I 

Geant4 default multiple scattering settings 


Geaiit4 Version 

Multiple Scattering Model 

9.1 

G4UrbanMscModel 

9.2 

G4UrbanMscModel2 

9.3 

G4UrbanMscModel92 

9.4 

G4UrbanMscModel93 

9.6 

G4UrbanMscModel95 

10.0 

G4UrbanMscModel 

10.1 

G4UrbanMscModel 


version. 

An algorithm combining multiple and single scattering is 
implemented in the G4WentzelVlModel class | |89t . 

A multiple scattering model based on Goudsmit-Saunderson 
calculations iizD> m has been available in Geant4 for the 
simulation of electron multiple scattering since version 9.3 
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| [90) . It is implemented in the G4GoudsmitSaundersonModel 
class. 

B. Multiple scattering in Geant4 prepackaged PhysicsLists 

Geant4 software documentation recommends the use 
of predehned classes in simulation applications, which im¬ 
plement selections of electromagnetic processes and mod¬ 
els for several particle types. These classes, derived from 
G4VPhysicsConstructor, are supplied within the physics lists 
package of the Geant4 toolkit; they are used in the prepack¬ 
aged PhysicsLists hosted in the same package or may be 
instantiated in user defined PhysicsLists. According to | [92) , 
they are “intensively validated”. 

Geant4 10.0 and 10.1 version encompass six generic elec¬ 
tromagnetic PhysicsConstructors, which are briefly described 
below, as well as a few specialized ones (e.g. for optical 
physics and for the interactions of polarized particles), which 
are outside the scope of this paper. 

The selection of electromagnetic processes and models 
implemented in the G4EmStandardPhysics PhysicsConstructor 
is described in the user documentation of Geant4 10.0 m as 
“default electromagnetic physics”. From an inspection of the 
source code it appears that for electrons the Urban multiple 
scattering model is activated up to 100 MeV, while the 
WentzelVI model is selected above that energy threshold. 
Single Coulomb scattering is activated in association with the 
WentzelVI model. A polar angle threshold is set at 180 de¬ 
grees. Step limit and range factor settings are kept unchanged 
with respect to the default implementations in the multiple 
scattering classes it instantiates. 

According to Geant4 user documentation 
G4EmStandardPhysics_optionl provides “fast but less 
accurate electron transport” due to the choice of the Simple 
method of step limitation by multiple scattering, reduced step 
limitation by the ionisation process and enabled ApplyCuts 
option. It uses G4UrbanMscModel for multiple scattering of 
electrons and positrons. From an inspection of the source 
code it appears that the model selections, energy and angle 
thresholds concerning multiple scattering are the same as in 
G4EmStandardPhysics. 

G4EmStandardPhysics_option2 is dehned in ID as “ex¬ 
perimental electromagnetic physics with disabled ApplyCuts 
option”. From an inspection of the source code it appears 
that the model selections, energy and angle thresholds con¬ 
cerning electron scattering scattering are the same as in 
G4EmStandardPhysics. 

Geant4 user documentation ID states that 
G4EmStandardPhysics_option3 selects electromagnetic 
physics for simulation with high accuracy due to 
UseDistanceToBoundary multiple scattering step limitation, 
reduced finalRange parameter of stepping function optimized 
per particle type, alternative model G4KleinNishinaModel 
for Compton scattering, Rayleigh scattering, and 
G4IonParameterisedLossModel for ion ionisation. From 
an inspection of the source code it appears that for electrons 
the default configuration of the G4eMultipleScattering process 
is selected, which in turn instantiates the Urban multiple 
scattering model. 


According to Geant4 user documentation |j9T|, the combi¬ 
nation of “best electromagnetic models for simulation with 
high accuracy” includes UseDistanceToBoundary multiple 
scattering step limitation, reduced finalRange parameter of 
stepping function optimized per particle type, low-energy 
sub-library models G4LivermorePhotoElectricModel, G4- 
LowEPComptonModel below 20 MeV, G4PenelopeGamma- 
ConversionModel below 1 GeV, G4PenelopeIonisationModel 
below 100 keV, and G4IonParameterisedLossModel for ion 
ionisation. This combination is implemented in G4Em- 
StandardPhysics_option4. From an inspection of the source 
code it appears that the model selections, energy and angle 
thresholds concerning multiple scattering are the same as in 
G4EmStandardPhysics. Some documentation of the perfor¬ 
mance of G4EmStandardPhysics_option4 with respect to ex¬ 
perimental data is available in [ |9^ . Despite our best efforts, we 
could not retrieve references in the literature supporting quan¬ 
titatively the statement that G4EmStandardPhysics_option4 
corresponds to “best electromagnetic models for simulation 
with high accuracy”. 

G4EmLivermorePhysics selects models for electrons and 
photons based on the EEDL | |94) and EPDL | [95] data libraries. 
From an inspection of the source code it appears that model 
selections, energy and angle thresholds, step limitation and 
RangeFactor settings concerning multiple scattering are the 
same as in G4EmStandardPhysics_option4. 

Two PhysicsConstructors specihc to simulations using 
single scattering and G4WentzelVIModel, named G4Em- 
StandardPhysics_SS and G4EmStandardPhysics_WVI respec¬ 
tively, are released for the first time in Geant4 10.1. 

IV. Simulation 
A. Simulation application 

A Geant4-based application was developed to simulate the 
electron backscattering experiments involved in the validation 
test. The application models the experimental scenarios, en¬ 
codes a selection of physics configurations corresponding to 
the options documented in section |I^ drives the simulation 
execution and assembles a set of signihcant observables pro¬ 
duced by the simulation into for further analysis. 

The geometry configuration reproduces the relevant features 
of the experimental setups described in 0- | [63| : a semi- 
inhnite or inhnite target of pure elemental composition and 
a hemispherical sensitive volume representing the electron 
detector. The target and the detector are surrounded by a very 
low density material (equivalent to galactic vacuum) to avoid 
contamination of the test results from spurious interactions. An 
electron beam impinges on the target. A sketch of the geometry 
conhguration is illustrated in Fig. Relevant parameters, such 
as the target shape and thickness, the angular acceptance of 
the detector, the energy and angular spread of the beam are 
retrieved from the experimental references 0-f63) whenever 
available. Electrons entering the sensitive volume are consid¬ 
ered detected, when their energy exceeds a preset threshold. 

As most references do not report the experimental conhgu¬ 
ration in detail, some assumptions are made in the simulation, 
when the corresponding parameters are not explicitly docu¬ 
mented: the target is assumed to be a cylinder of thickness and 


TABLE III 

Geant4 versions subject to test 

Version Identifier Patehed Geant4 Version 
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Fig. 2. A sketch of the geometrical setup implemented in the simulation, 
produced by Geant4 visualization package. Red lines represent backscattered 
electrons; green lines correspond to photons escaping from the target. 


diameter larger than the incident electron range retrieved from 
the ESTAR | |9^ database, the sensitive detector volume is 
assumed to cover the whole backward hemisphere, a detection 
threshold of 50 eV is applied, consistent with common exper¬ 
imental practice to exclude secondary electrons, the detection 
efficiency above this threshold is assumed to be 100%, the 
electron beam is assumed to be monochromatic and incident 
at 90°. These assumptions correspond to modeling an infinite 
target, whose geometrical characteristics would not affect the 
resulting backscattering measurement, and to presuming that 
the backscattering values reported in the experimental refer¬ 
ences were previously corrected for geometrical acceptance 
and detection efficiency as appropriate. 

The backscattering fraction is calculated as the ratio be¬ 
tween the number of events with at least one detected electron 
and the number of primary electrons. Detected electrons may 
be backscattered primary electrons or secondary electrons with 
energy above the preset threshold. Events with more than one 
detected electron contributed to the backscattering fraction as 
a single count. 

The physics configurations concerning electron scattering 
evaluated in this paper are summarized in Table [I^ The 
class names reported in this table appear as in Geant4 10.0; 
different settings in Geant4 10.0 and 10.1 are identified by 
the respective version number in parentheses. The UML (Uni¬ 
fied Modeling Language) class diagram in Eig. illustrates 
the PhysicsConstructors involved in the application and their 
relationship with classes in Geant4 kernel responsible for the 
configuration of physics selections. 

B. Simulation Production 

Simulations are produced with six Geant4 versions released 
between December 2007 and December 2014. The selection 
reflects the study of Geant4 energy deposited by electrons 
documented in Q, with the addition of Geant4 version 10.0 
and 10.1, which were first released after the publication of that 
paper, and the exclusion of Geant4 version 9.5, which exhib¬ 
ited a very similar behaviour to version 9.6 in Q. Correction 


9.1 

9.1p03 

9.2 

9.2p04 

9.3 

9.3p02 

9.4 

9.4p04 

9.6 

9.6p03 

10.0 

10.0p03 

10.1 

not pertinent 


patches to these versions released by October 2014 are applied 
on top of the original versions. Eor convenience, the Geant4 
versions evaluated in this study are identified through their 
original version number; the corresponding patched versions 
used in the validation test are listed in Table Hn] 

The simulations were executed on workstations running the 
Scientific Linux 6 operating system with the gcc version 4.4.7 
compiler and on computers running MacOS 10.8.5 (Mountain 
Lion) operating system with clang 3.3. The same Geant4 
versions and application code were installed on all machines. 

IThe simulation configurations corresponding to the exper¬ 
imental test cases were encoded in “macro” files handled by 
Geant4 user interface. Configurations corresponding to differ¬ 
ent physics options for the same experimental scenario were 
automatically generated based on a master configuration file to 
minimize the possibility of accidental encoding errors, which 
could be the source of systematic effects in the validation 
results. All simulation configuration files were kept under 
version control to ensure the reproducibility of results. 

The physics configurations produced in the Geant4 versions 
subject to evaluation are listed in Table |IV] Simulations 
using the family of Urban configurations and the default 
single Coulomb scattering configuration were produced in all 
versions, while those using the WentzelVI and Goudsmit- 
Saunderson models were produced with Geant4 version 9.3 
and later, consistent with their first release. Simulations with 
Coulomb scattering settings different from the default ones 
(namely with the “6* limit” parameter set to zero) were 
produced with Geant4 version 10.1. 

Simulations using predefined electromagnetic PhysicsCon¬ 
structors were produced with Geant4 versions 9.6, 10.0 
and 10.1, which reflect the recommendations to experimen¬ 
tal users issued in the most recent versions of Geant4 
at the time of writing this paper. Simulations using the 
G4EmStandardPhysics_WVI and G4EmStandardPhysics_SS 
predefined electromagnetic PhysicsConstructors were limited 
to Geant4 version 10.1, where these classes have been first 
released. 

The secondary production threshold was set to 1 
(expressed in terms of particle range) in all simulations, with 
the exception of test cases with primary electron beam energy 
above 1 MeV, for which it was raised to 10 to reduce the 
execution time compatible with the available computational 
resources. 

A user-defined step limitation was applied in the simulations 
corresponding to the Urban and UrbanB configurations; the 
step limit was set to the same value as the secondary produc¬ 
tion threshold setting. 




























Fig. 3. UML (Unihed Modeling Language) class diagram illustrating the physics configuration in the simulation: simulation application classes (white), 
Geant4 kernel classes (dark grey) and PhysicsConstructor classes released in the Geant4 physics_lists package (light grey). 


For each simulation run, the number of generated events was 
determined to ensure that the statistical errors on the simulated 
backscattering fraction was smaller or at most comparable to 
the experimental uncertainty reported in the corresponding 
reference paper or in similar experimental configurations, 
when the associated reference did not document uncertainties. 
Based on these considerations, the number of events in a 
single run varied between 10000 and 500000, depending on 
the corresponding experimental configuration. 


V. Data Analysis 

The data analysis addresses various issues related to the val¬ 
idation of Geant4-based simulation of electron backscattering: 
the evaluation of the capability of Geant4 to produce results 
consistent with experiment, the comparison of the simulation 
accuracy achievable with different Geant4 multiple and single 
scattering configurations in the user application, the correlation 
between the validity of simulated electron backscattering and 
energy deposition, and the evolution of the results over a series 
of Geant4 versions. Appropriate statistical methods are applied 
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TABLE II 

Multiple and single scattering configurations evaluated in the electron backscattering validation test 


Configuration 

Description 

Process class 

Model class 

StepLimitiype 

RangcFactor 

Urban 

Urban model, user step limit 

G4eMultipleScattering 

G4UrbanMscModel 

default 

default 

UrbanB 

Urban model, user step limit 

G4eMultipleScattering 

G4UrbanMscModel 

DistanceToBoundary 

default 

UrbanBRF 

Urban model 

G4eMultipleScattering 

G4UrbanMscModel 

DistanceToBoundary 

0.01 

GSBRF 

Goudsmit-Saunderson 

G4eMultipleScattering 

G4GoudsmitSaundersonModel 

DistanceToBoundary 

0.01 

WentzelBRF 

WentzelVI model 

G4eMultipleScattering 

G4WentzelVIModel 

DistanceToBoundary 

0.01 

WentzelBRFP 

WentzelVI, angle limit 

G4eMultipleScattering 

G4WentzelVIModel 

DistanceToBoundary 

0.01 

Coulomb 

Single scattering, default 

G4CoulombScattering 

G4eCoulombScatteringModel 

default 

default 

CoulombP 

Single scattering, 
with angle limit (10.1) 

G4CoulombScattering 

G4eCoulombScatteringModel 

default 

default 



PhysicsConstructor class 



EmLivermore 


G4EmLivermorePhysics 

DistanceToBoundary 

0.01 

EmStd 

Predefined 

G4EmStandardPhysics 

default 

default 

EmOptl 

electromagnetic 

G4EmStandardPhy sics_option 1 

default 

default 

EmOptl 

physics 

G4EmStandardPhysics_option2 

default 

default 

EmOpt3 

selections 

G4EmStandardPhysics_option3 

DistanceToBoundary 

default 

EmOpt4 


G4EmStandardPhysics_option4 

DistanceToBoundary (10.0) 

0.01 (10.0) 





SafetyPlus (10.1) 

0.02 (10.1) 

EmSS 


G4EmStandardPhysics_SS (lO.l) 

default 

default 

EmWVI 


G4EmStandardPhysics WVI (10.1) 

default 

default 


TABLE IV 

Physics configurations in the simulation production 


Versions 

Scattering 

Electrons 

Photons 

9.1-10.1 

Urban 

UrbanB 

UrbanBRF 

Coulomb 

EEDL, Standard, Penelope 
EEDL 

EEDL 

EEDL 

EPDL 

9.3-10.1 

GSBRF 

WentzelBRF 

WentzelBRFP 

EEDL 

EPDL 

9.6-10.1 

EmLivermore 

EmStd 

EmOptl 

EmOpt2 

EmOpt3 

EmOpt4 

Embedded 

Embedded 

10.1 

EmSS 

EmWVI 

Embedded 

Embedded 

CoulombP 

EEDL 

EPDL 


to analyze the data pertinent to each problem. 


A. Analysis method 

The experimental data sample retrieved from the literature 
is dominated by low energy measurements (mostly below a 
few tens keV); this intrinsic characteristic of the data set 
would affect the ability to discern the energy dependence of 
the capabilities of Geant4 models from an analysis of their 
compatibility with experiment over the whole data sample, 
as the the outcome of the statistical tests would be mainly 
determined by the weight of the low energy component. 
Therefore, the experimental data sample was partitioned into 
three energy ranges, identified in the following as “low”, 
“intermediate” and “high” energy, respectively, to evaluate 
the capabilities of the examined models as a function of the 
electron energy. 

The grouping of data in three energy intervals is empirically 
based on the behaviour observed in the plots reporting sim¬ 
ulation results and experimental data. To mitigate the risk of 
biased conclusions due to the empirical nature of its definition, 
it was verified that the general conclusions of the validation 


test reported in the following sections are stable with respect 
to small variations of the definition of low, intermediate and 
high energies (e.g. 10 keV instead of 20 keV, or 75 keV 
instead of 100 keV). This verification was done by performing 
the whole analysis procedure over four different definitions of 
the three energy groups: although the numerical values of the 
test statistics were slightly different for different partitioning 
of the data sample, the global conclusions of the validation 
tests were insensitive to how the three energy ranges were 
defined. The results reported in this paper data correspond to 
grouping the data between 1 and 20 keV, between 20 and 
100 keV and above 100 keV. Data below 1 keV are excluded 
from the analysis, since all the examined models appear inca¬ 
pable of reproducing the experimental measurements below 
this threshold. The apparent absence of any backscattering 
occurrence below this energy, concerning some models, hints 
at an intrinsic limitation of their functionality. 

In the context of the data analysis a test case is defined 
by a set of experimental data, associated with a given target 
composition, within a given energy range, performed by a 
given research group. The previously mentioned energy in¬ 
tervals encompass 137, 112 and 57 test cases, respectively. 

The statistical analysis of compatibility between simulation 
and experiment is articulated over two stages. 

The first stage of the statistical analysis consists of eval¬ 
uating the compatibility between simulation and experiment 
in single test cases. The compatibility between simulated and 
experimental data in a single test case is determined by two- 
sample goodness-of-fit tests. The details of this component of 


the analysis are described in section V-B 


In the second stage of the analysis categorical statistical 
tests determine whether the differences in compatibility with 
experiment observed across the various categories of Geant4 
models can be explained only by chance, or should be inter¬ 
preted as deriving from intrinsic behavioural characteristics. 

The approach adopted in the categorical analysis is con¬ 
nected to the physics configuration that characterizes how the 
data samples were produced, namely whether the data subject 
to test are unrelated or to some extent related. Simulations 

















using different multiple scattering models produce unrelated 
samples, as the different modeling options available in Geant4 
intend to implement distinct conceptual alternatives in the 
treatment of multiple scattering. Data samples deriving from 
simulations that differ only for a secondary option (e.g. a pa¬ 
rameter setting), but are based on the same multiple scattering 
model, are to some extent related; this is the case, for instance, 
of the three configurations based on the Urban model and the 
two conhgurations based on the WentzelVl model listed in 
Table It is also the case of simulations based on single 
Coulomb scattering and the WentzelVl multiple scattering 
model, since the latter incorporates the former. Statistical tests 
pertinent to independent or related data samples are applied 
as appropriate in this stage of the analysis. 

Finally, a statistical analysis is performed to evaluate 
whether a correlation can be established between the compat¬ 
ibility with experiment of electron backscattering and energy 
deposition simulated by Geant4. 

The significance level for the rejection of the null hypothesis 
is set at 0.01 for all tests, unless otherwise specihed. 

The statistical data analysis reported in the following sec¬ 
tions exploits the Statistical Toolkit | |97| , | [98) for goodness- 
of-fit testing and R | |99) for the analysis of contingency tables 
and of correlation. 


B. Evaluation of individual test cases 

This stage of the analysis determines whether the experi¬ 
mental and simulated data distributions associated with each 
test case are consistent with deriving from the same parent 
distribution. 

The generally poor documentation of experimental errors in 
the experimental references, discussed in section II-A prevents 
the use of the test ]100) , which involves measurement 
uncertainties explicitly. Therefore goodness-of-fit tests based 
on the empirical distribution function, where uncertainties do 
not appear directly in the calculation of the test statistic, 
were used in the analysis. Four independent goodness-of-ht 
tests were performed for each test case to mitigate the risk 
of introducing systematic effects in the validation process 
due to peculiarities of the mathematical formulation of the 
test statistic: the Anderson-Darling P01[ , |102|, Cramer-von 
Mises P03|, P04| , Kolmogorov-Smirnov P05) , P6l and 
Watson ll07| tests. 

The null hypothesis for each goodness-of-fit test is that the 
simulated and experimental distributions derive from the same 
parent distribution. The outcome of the test is classihed as 
“fail”, if the null hypothesis is rejected, as “pass” otherwise. 

For convenience, the “efficiency” of a Geant4 simulation 
conhguration is dehned as the fraction of test cases in which 
a goodness-of-ht test does not reject the null hypothesis at 0.01 
level of signihcance. This variable quantihes the capability of 
that simulation conhguration to produce results statistically 
consistent with experiment over the whole set of test cases 


pertinent to one of the energy ranges dehned in section V-A 


Despite some qualitatively visible inconsistencies in the 
experimental data, no attempt was made to exclude the con¬ 
cerned data sets from the analysis, nor to remove single 


TABLE V 

CONHOURATION OF CONTINGENCY TABLES FOR THE EVALUATION OF 
UNRELATED SIMULATION CONFIGURATIONS 



Category A 

Category B 

Pass 

Npassyi 

^passB 

Fail 

N/ailA 

N/aiiS 


outliers, since the poor documentation of experimental errors 
prevents a proper evaluation of the consistency of the various 
measurements. The possible leftover presence of experimental 
data affected by systematic errors may artihcially lower the 
efficiency associated with a given Geant4 model: therefore, 
caution should be applied to interpreting the efficiency val¬ 
ues reported in this papers as absolute estimates of Geant4 
simulation capabilities. Nevertheless, since all Geant4 physics 
conhgurations are exposed to the same test cases, the possible 
inclusion of experimental data samples exhibiting systematic 
effects would marginally affect the conclusions regarding the 
comparison of their compatibility with experiment. 


C. Evaluation of unrelated data samples 

This component of the analysis evaluates the difference in 
compatibility with experiment across independent categories 
of data, e.g. associated with two different multiple scattering 
models. It receives its input from the results of goodness-of-ht 
tests described in section IV-BI 

The differences in the behavior of the two categories are 
quantihed by means of contingency tables, which are built by 
counting the number of test cases identihed as “fail” or “pass” 
in section V-B Their conhguration is illustrated in Table 

In the analysis of contingency tables the null hypothesis is 
that there is no relationship between the two categories; in 
physical terms it means that the two simulation conhgurations 
under examination are equivalent regarding the compatibility 
with experiment of their respective outcome. 

A variety of tests is applied to determine the statistical 
signihcance of the difference between the t wo ca tegories of 
data subject to evaluation: Pearson’s test 1 108j (when the 
number of entries in each cell of the table is sufficiently 
large to justify its applicability), Fisher’s exact test P09l and 
a variety of unconditional exact tests. The use of different 
tests mitigates the risk of introducing systematic effects in 
the validation results due to peculiarities in the mathematical 
formulation of the test statistic. 

Fisher’s exact test is widely used in the analysis of contin¬ 
gency tables; although it is based on a model in which both 
the row and column sums are hxed in advance, which seldom 
occurs in experimental practice, it remains valid when the row 
or column totals, or both, are not fixed, but in these cases it 
tends to be conservative, yielding a larger p-value than the true 
significance of the test ing. 

Unconditional tests, such as Barnard’s test |111| , Boschloo’s 
test GH) and Suissa and Shuster’s GH calculation of a 
Z-pooled statistic, are deemed more powerful than Fisher’s 
exact test in some configurations of 2x2 contingency tables 
GB). GI3’ but they are computationally more intensive. 
They yield consistent results in the context of the validation 
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TABLE VI 

CONHOURATION OF CONTINGENCY TABLES FOR THE EVALUATION OF 
RELATED SIMULATION CONFIGURATIONS 



Category A: Pass 

Category A: Fail 

Category B: Pass 
Category B: Fail 

Npass A, passB 

Npass A, fail B 

^fail A, pass B 
^fail A, fail B 


process described in this paper; the p-values of two of such 
tests are reported among the results of the validation analysis 
in section 1^ 


D. Evaluation of related data samples 

This component of the analysis evaluates the difference 
in compatibility with experiment across related categories of 
data. It receives its input from the results of goodness-of-fit 
tests. 

This analysis method is applied to two situations: when 
each subject (e.g. a physics configuration in the simulation) 
serves in both categories being evaluated (e.g. two Geant4 
versions), and when one examines two closely related subjects 
(e.g. simulations differing for a secondary feature of a multiple 
scattering model, while all the other physics configuration 
settings are identical). 

Appropriate 2x2 contingency tables are built for this pur¬ 
pose, based on the results of the goodness-of-ht tests doc¬ 
umented in section |V-B[ for each goodness-of-fit test, they 
report on one diagonal the number of test cases where both 
categories subject to evaluation “pass” or “fail” the test, and on 
the other diagonal the number of test cases where one category 
“passes” the yf test, while the other one“fails”. An example 


of their configuration is shown in Table VI 


McNemar’s test m is applied to the analysis of related 
data samples. This test focuses on the significance of the 
discordant results, i.e. the number of test cases where one 
category “passes” a goodness-of-fit test and the other one 
“fails”. The null hypothesis for McNemar’s test is that the 
proportion of discordant results is the same in the two cells 
corresponding to “pass-fail” or “fail-pass” associated with the 
two categories subject to test. 

The calculation of McNemar’s test is performed using either 
the asymptotic distribution or the binomial distribution 
ifn?! : the latter is also known as “McNemar exact test”. Yates’ 
P 18[ continuity correction may be applied to the calculation 
of the statistic to account for cells with a small number of 
entries. According to 1 1 19) , McNemar’s test uncorrected for 
continuity is more powerful than the exact test, and performs 
well even when the number of discordant pairs is as low as 
6, while both the exact test and the corrected McNemar’s test 
are conservative. The results reported in this paper concern 
McNemar exact test, as most of the analyzed tables involved 
a small number of entries in some cells, which prevented the 
calculation of the statistic. 


VI. Results 

The analysis of electron backscattering simulation is fo¬ 
cused on the quantihcation of a few salient features, based 
on the considerations discussed in the previous sections: 


• the quantihcation of the capability of each physics con- 
hguration to simulate electron backscattering compatible 
with experiment; 

• its evolution with Geant4 versions, with emphasis on 
quantifying the capabilities of the latest versions at the 
time of writing this paper, to which the interest of 
experimental users is more generally directed; 

• the identihcation of the physics conhgurations and 
Geant4 versions achieving the highest compatibility with 
experiment; 

• the role played by single and multiple scattering in 
contributing to the accuracy and the computational speed 
of the simulation. 

The following sections document first an overview of the 
main features of the statistical results, followed by detailed 
discussion of each Geant4 model configuration subject to 
evaluation. 

The large number of experimental test cases, Geant4 physics 
modeling options and versions evaluated in the validation tests 
makes a full graphical illustration of the results impractical 
within the scope of a journal publication. A sample of plots, 
which span a wide range of electron beam energies, target 
atomic numbers, Geant4 models and versions, complement the 
outcome of the statistical analysis with a qualitative illustration 
of the results. 


A. Model Compatibility with Experiment 

The efficiencies resulting from goodness-of-fit tests are 
summarized in Table VII for all the physics conhgurations and 
Geant4 versions evaluated in this paper. One can observe that 
the four tests produce consistent outcomes regarding the rejec¬ 
tion of the null hypothesis of equivalence between simulated 
and experimental distributions. Therefore, for convenience, the 
categorical analyses reported in the following sections are 
performed only based on the outcome of the Anderson-Darling 
test, unless otherwise specihed. 

The results listed in Table IVIII show that Geant4 multi¬ 
ple scattering models fail to reproduce experimental electron 
backscattering data in the lower energy end, while the single 
scattering approach adopted in the Coulomb and WentzelBRF 
conhgurations retains the capability to describe lower energy 
data, although with reduced efficiency with respect to the 
higher energy range. At energies above 100 keV the highest 
efficiency is achieved with the Urban conhguration in Geant4 
9.1 in the context of a condensed path scheme, and with the 
Coulomb scattering model in Geant4 10.0 in the context of 
simulating individual scattering occurrences. 

Evaluations specihc to each Geant4 model, or family of 
models, are detailed in the following sections. 


B. Effect of different Geant4 electron physics models 

The fraction of backscattered electrons produced in the 
simulation is mainly determined by multiple scattering; a small 
contribution to the number of electrons reaching the detector 
consists of secondary particles produced in the ionization of 
target atoms, which escape the target volume. 
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TABLE VII 

Efficiency of physics configurations for Geant4 versions 9.1 to 10.1 


Configuration 

Geant4 


1-20 keV 



20-100 keV 



> lOOfeeU 



version 

AD 

CvM 

KS 

Watson 

AD 

CvM 

KS 

Watson 

AD 

CvM 

KS 

Watson 

Urban 

9.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.10±0.03 

0.10±0.03 

0.10±0.03 

0.10±0.03 

0.79±0.05 

0.79±0.05 

0.77±0.05 

0.79±0.05 

Urban 

9.2 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.03±0.02 

0.03±0.02 

0.03±0.02 

0.03±0.02 

0.79±0.05 

0.77±0.05 

0.82±0.05 

0.88±0.04 

Urban 

9.3 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.09±0.03 

0.09±0.03 

0.09±0.03 

0.09±0.03 

0.74±0.06 

0.74±0.06 

0.72±0.06 

0.74±0.06 

Urban 

9.4 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.10±0.03 

0.10±0.03 

0.10±0.03 

0.10±0.03 

0.56±0.06 

0.56±0.06 

0.56±0.06 

0.61±0.06 

Urban 

9.6 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.17±0.04 

0.17±0.04 

0.17±0.04 

0.17±0.04 

0.68±0.06 

0.68±0.06 

0.68±0.06 

0.82±0.05 

Urban 

10.0 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.11±0.04 

0.11 ±0.04 

0.11±0.04 

0.09±0.04 

Urban 

10.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.07±0.04 

0.07±0.04 

0.07±0.04 

0.09±0.04 

UrbanB 

9.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.10±0.02 

0.10±0.02 

0.10±0.02 

0.10±0.02 

0.79±0.05 

0.79±0.05 

0.77±0.05 

0.79±0.05 

UrbanB 

9.2 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.03±0.02 

0.03±0.02 

0.03±0.02 

0.03±0.02 

0.79±0.05 

0.77±0.05 

0.82±0.05 

0.88±0.04 

UrbanB 

9.3 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.11±0.04 

0.11 ±0.04 

0.11±0.04 

0.11±0.04 

UrbanB 

9.4 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.07±0.04 

0.07±0.04 

0.07±0.04 

0.07±0.04 

UrbanB 

9.6 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.05±0.03 

0.05±0.03 

0.05±0.03 

0.07±0.04 

UrbanB 

10.0 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.07±0.04 

0.07±0.04 

0.07±0.04 

0.07±0.04 

UrbanB 

10.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.09±0.04 

0.07±0.04 

0.09±0.04 

0.11±0.04 

UrbanBRF 

9.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.05±0.03 

0.05±0.03 

0.05±0.03 

0.07±0.04 

UrbanBRF 

9.2 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.02±0.02 

0.02±0.02 

0.02±0.02 

0.04±0.03 

UrbanBRF 

9.3 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.07±0.04 

0.07±0.04 

0.07±0.04 

0.07±0.04 

UrbanBRF 

9.4 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.07±0.04 

0.07±0.04 

0.07±0.04 

0.07±0.04 

UrbanBRF 

9.6 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.07±0.04 

0.07±0.04 

0.07±0.04 

0.07±0.04 

UrbanBRF 

10.0 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.07±0.04 

0.07±0.04 

0.07±0.04 

0.07±0.04 

UrbanBRF 

10.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.09±0.04 

0.09±0.04 

0.09±0.04 

0.09±0.04 

Coulomb 

9.1 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.16±0.03 

0.16±0.03 

0.17±0.04 

0.18±0.04 

0.60±0.06 

0.60±0.06 

0.60±0.06 

0.68±0.06 

Coulomb 

9.2 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.04±0.02 

0.04±0.02 

0.04±0.02 

0.04±0.02 

0.61±0.06 

0.61±0.06 

0.65±0.06 

0.65±0.06 

Coulomb 

9.3 

0.19±0.03 

0.18±0.03 

0.18±0.03 

0.20±0.03 

0.08±0.03 

0.08±0.03 

0.08±0.03 

0.09±0.03 

0.63±0.06 

0.63±0.06 

0.63±0.06 

0.67±0.06 

Coulomb 

9.4 

0.03±0.02 

0.03±0.02 

0.04±0.02 

0.04±0.02 

0.22±0.04 

0.21±0.04 

0.21±0.04 

0.21±0.04 

0.68±0.06 

0.68±0.06 

0.72±0.06 

0.74±0.06 

Coulomb 

9.6 

0.48±0.04 

0.48±0.04 

0.48±0.04 

0.46±0.04 

0.42±0.05 

0.42±0.05 

0.42±0.05 

0.41±0.05 

0.79±0.05 

0.77±0.05 

0.72±0.06 

0.81±0.05 

Coulomb 

10.0 

0.49±0.04 

0.49±0.04 

0.48±0.04 

0.48±0.04 

0.40±0.05 

0.39±0.05 

0.39±0.05 

0.38±0.05 

0.79±0.05 

0.77±0.05 

0.79±0.05 

0.89±0.04 

Coulomb 

10.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.02 

< 0.02 

< 0.02 

< 0.02 

CoulombP 

10.1 

0.47±0.04 

O 

o 

d 

0.47±0.04 

o 

o 

d 

0.45±0.05 

0.44±0.05 

0.44±0.05 

0.43±0.05 

0.81±0.05 

0.81±0.05 

0.77±0.05 

0.88±0.04 

GSBRF 

9.3 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.07±0.03 

0.07±0.03 

0.07±0.03 

0.07±0.03 

GSBRF 

9.4 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.07±0.03 

0.07±0.03 

0.07±0.03 

0.07±0.03 

GSBRF 

9.6 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.58±0.06 

0.58±0.06 

0.60±0.06 

0.58±0.06 

GSBRF 

10.0 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.58±0.06 

0.56±0.06 

0.58±0.06 

0.54±0.06 

GSBRF 

10.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.39±0.06 

0.40±0.06 

0.39±0.06 

0.39±0.06 

WentzelBRF 

9.3 

0.18±0.03 

0.18±0.03 

0.19±0.03 

0.20±0.03 

0.09±0.03 

0.09±0.03 

0.09±0.03 

0.10±0.03 

0.60±0.06 

0.60±0.06 

0.60±0.06 

0.65±0.06 

WentzelBRF 

9.4 

0.02±0.01 

0.02±0.01 

0.02±0.01 

0.04±0.02 

0.21±0.04 

0.20±0.04 

0.20±0.04 

0.20±0.04 

0.61±0.06 

0.60±0.06 

0.60±0.06 

0.68±0.06 

WentzelBRF 

9.6 

0.46±0.04 

0.46±0.04 

0.46±0.04 

0.46±0.04 

0.44±0.05 

0.43±0.05 

0.43±0.05 

0.42±0.05 

0.79±0.05 

0.77±0.05 

0.75±0.06 

0.81±0.05 

WentzelBRF 

10.0 

0.49±0.04 

0.48±0.04 

0.48±0.04 

0.49±0.04 

0.44±0.05 

0.43±0.05 

0.43±0.05 

0.42±0.05 

0.81±0.05 

0.79±0.05 

0.82±0.05 

0.88±0.04 

WentzelBRF 

10.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.42±0.06 

0.42±0.06 

0.42±0.06 

0.46±0.06 

WentzelBRF? 

9.3 

0.02±0.01 

0.02±0.01 

0.02±0.01 

0.03±0.02 

0.04±0.02 

0.04±0.02 

0.04±0.02 

0.04±0.02 

0.33±0.06 

0.33±0.06 

0.33±0.06 

0.37±0.06 

WentzelBRF? 

9.4 

0.02±0.01 

0.02±0.01 

0.02±0.01 

0.02±0.01 

0.03±0.02 

0.03±0.02 

0.03±0.02 

0.03±0.02 

0.21±0.05 

0.21±0.05 

0.21±0.05 

0.21±0.05 

WentzelBRF? 

9.6 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.30±0.05 

0.28±0.05 

0.30±0.05 

0.33±0.05 

WentzelBRF? 

10.0 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.44±0.06 

0.42±0.06 

0.44±0.06 

0.46±0.06 

WentzelBRF? 

10.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

O.OliO.Ol 

0.42±0.06 

0.42±0.06 

0.42±0.06 

0.46±0.06 

EmLivermore 

9.6 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.07±0.04 

0.05±0.03 

0.07±0.04 

0.07±0.04 

EmLivermore 

10.0 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.05±0.03 

0.05±0.03 

0.05±0.03 

0.05±0.03 

EmLivermore 

10.1 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

< 0.01 

0.07±0.04 
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As Geant4 encompasses different models of electron ion¬ 
ization, their effect on the validation of the simulation of 
electron backscattering has been evaluated. For this purpose 
simulations where executed with the Urban multiple scattering 
conhguration associated with different Geant4 electron physics 
models: those based on the EEDL | |94) evaluated data library 
|fT20|-|fT^ , those encompassed in the Geant4 Standard elec¬ 
tromagnetic packag e |123[ and those reengineered in Geant4 
from the Penelope |69| Monte Carlo code. 


The efficiency obtained with the three different conhgura- 
tions is listed in Table VIII for the Geant4 versions subject to 
evaluation: it derives from the results of the Anderson-Darling 
goodness-of-ht test comparing simulated and experimental 
backscattered electron distributions. One can observe that the 
results associated with different electron physics models are 
very similar; a categorical analysis comparing the results of 
the Standard and Penelope models with those based on EEDL 
fails to reject the hypothesis of equivalent behaviour with 0.05 
significance. Both Eisher’s and Barnard’s exact tests lead to the 
same conclusion. 


Erom this analysis one can infer that the electron backscat¬ 
tering test studied in this paper is insensitive to how electron 
interactions, apart from multiple scattering, are modeled in 
Geant4. 


The results reported in the following sections are obtained 
using Geant4 electron interaction models based on EEDL. 

The interactions of secondary photons were modeled based 
on the EPDL | [95 ) evaluated data library in all simulations, 
based on validation studies 1 124|-| T27) : they play a negligible 
role in the validation test, as they could only affect the 
estimated electron backscattering fraction as a contamination 
originating from secondary electrons they produce. No such 
contamination was observed in the simulated data sample 
subject to analysis. 


TABLE VIII 

Efficiency for the Urban multiple scattering configuration 

WITH different ELECTRON PHYSICS MODELS 


Energy (keV) 

Version 

EEDL 

Standard 

Penelope 


9.1 

< 0.01 

< 0.01 

< 0.01 


9.2 

< 0.01 

< 0.01 

< 0.01 


9.3 

< 0.01 

< 0.01 

< 0.01 

1-20 

9.4 

< 0.01 

< 0.01 

< 0.01 


9.6 

< 0.01 

< 0.01 

< 0.01 


10.0 

< 0.01 

< 0.01 

< 0.01 


10.1 

< 0.01 

< 0.01 

< 0.01 


9.1 

O.lOibO.03 

0.11±0.03 

0.07±0.02 


9.2 

0.03±0.02 

0.06±0.02 

0.03±0.02 


9.3 

0.09±0.02 

0.04±0.02 

0.07±0.02 

20-100 

9.4 

O.lOibO.03 

0.08±0.03 

0.09±0.03 


9.6 

0.17±0.04 

0.15±0.03 

0.12±0.03 


10.0 

< 0.01 

< 0.01 

< 0.01 


10.1 

< 0.01 

< 0.01 

< 0.01 


9.1 

0.79±0.05 

0.75±0.06 

0.77±0.05 


9.2 

0.79±0.05 

0.81±0.05 

0.74±0.06 


9.3 

0.74±0.06 

0.65±0.06 

0.67±0.06 

>100 

9.4 

0.58±0.06 

0.61±0.06 

0.61±0.06 


9.6 

0.68±0.06 

0.72±0.06 

0.68±0.06 


10.0 

0.11±0.04 

0.09±0.04 

0.09±0.04 


10.1 

0.07±0.04 

0.07±0.04 

0.09±0.04 



Fig. 4. Fraction of electrons backscattered from a carbon target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 9.1 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 


C. Geant4 Urban models 


The fraction of backscattered electrons produced with the 
Urban, UrbanB and UrbanBRE conhgurations is illustrated in 
Eigs. 4p7 for Geant4 versions 9.1 to 10.0. Plots concerning 
Geant4 10.1 are omitted, as they look very similar to those 
produced with version 10.0. In these hgures and the following 
ones the labels associated with experimental data encode the 
name of the hrst author and the publication year of the 
reference they derive from, thus allowing the tracebility of the 
experimental data points appearing in the plots. In all hgures, 
experimental errors are plotted whenever they are documented 
in the associated papers; in some cases they may be smaller 
than the marker size. The statistical uncertainties of simulated 
data are not drawn for better clarity of the plots; in most cases 
they are smaller than the marker size. 

As a qualitative appraisal, one can observe that none of the 
three conhguration variants is capable of describing electron 
backscattering accurately below a few tens of keV. The sim¬ 
ulations adopting the Urban and UrbanB multiple scattering 
conhgurations exhibit different characteristics of compatibility 
with experimental data along the evolution of Geant4 from 
version 9.1 to 10.1, while simulations adopting the UrbanBRE 
options appear to behave consistently over all the examined 
versions. 

Eor energies above a few tens of keV the evolution of 
the compatibility with experiment of simulations using the 
Urban conhguration seems to follow a similar pattern to that 
observed in the validation of Geant4 simulation of deposited 
energy in |[I] over Geant4 versions 9.1 to 9.6. Visibly different 
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Fig. 5. Fraction of electrons backscattered from a silicon target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 9.1 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 


Fig. 7. Fraction of electrons backscattered from a tin target as a function of 
the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 9.1 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 
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Fig. 6. Fraction of electrons backscattered from a copper target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 9.1 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 


Fig. 8. Fraction of electrons backscattered from a gold target as a function of 
the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 9.1 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 
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results are obtained with Geant4 version 10.0, which mani¬ 
festly worsens the capability of the simulation to reproduce 
experimental data. This behaviour persists in Geant4 10.1. A 
qualitative illustration of the evolution of simulations using 


the Urban configuration is summarized in Figs. 18 and 19 


These empirical observations are quantitatively supported by 
the results of goodness-of-fit tests in Table |VII| The highest 
compatibility with experiment is achieved with the Urban 
multiple scattering configuration in the earlier Geant4 versions 
among those evaluated in this paper. 

The evolution of the capability of the Urban multiple 
scattering configuration to produce backscattering simulations 
compatible with experiment is quantihed in Table IX the sec¬ 
ond column reports the p-values of McNemar’s test comparing 
the compatibility with experiment of the Urban configuration 
in Geant4 9.1, which according to Table |VII| produces the 
highest efficiency, with later versions. The hypothesis of 
equivalent compatibility with experiment is rejected with 0.01 
significance for the Urban conhguration in Geant4 versions 
9.4, 10.0 and 10.1. These results concern the energy range 
above 100 keV; the statistical comparison of lower energy 
results is less relevant due to the reduced ability of the Urban 
multiple scattering conhguration to reproduce experimental 
measurements below a few tens of keV. 
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TABLE IX 

P- VALUES OF MCNEMAR EXACT TEST COMPARING THE COMPATIBILITY 
WITH EXPERIMENT OF THE URBAN CONFIGURATION IN GEANT4 9.1 WITH 
LATER GEANT4 VERSIONS AND WITH VARIANTS OF THE URBAN 
CONFIGURATION 


Geant4 

Version 

Urban 

Configuration 

UrbanB UrbanBRF 

9.1 


1.000 

<0.001 

9.2 

1.000 

1.000 

<0.001 

9.3 

0.453 

<0.001 

<0.001 

9.4 

0.002 

<0.001 

<0.001 

9.6 

0.070 

<0.001 

<0.001 

10.0 

<0.001 

<0.001 

<0.001 

10.1 

<0.001 

<0.001 

<0.001 


The backscattering fraction resulting from the UrbanBRF 
options appears to be very small in all test cases and differs 
signihcantly from experimental data. In depth inspection of 
the results shows that the fraction of detected electrons mainly 
consists of secondary particles produced by ionization of target 
atoms. These qualitative observations are conhrmed by the 
results of goodness-of-ht tests listed in Table VII the number 
of test cases that fail to reject the hypothesis of compatibility 
between experimental and simulated backscattering fractions is 
very small for the UrbanBRF multiple scattering conhguration 
at all energy intervals. 

The UrbanB conhguration produces identical results to the 
Urban one in Geant4 versions 9.1 and 9.2, while it approaches 
the behaviour of the UrbanBRF conhguration in later Geant4 
versions. Both conhgurations apply the DistanceToBoundary 
step limit type in the multiple scattering algorithm, while they 
differ in the textitRangeFactor setting, which is not modihed 
with respect to its default value in the UrbanB conhgura¬ 
tion and is assigned the value of 0.01 in the UrbanBRF 
conhguration. The observed evolution in their compatibility 
with experiment hints at an effect of the multiple scattering 


Fig. 9. Fraction of electrons backscattered from a silver target as a function of 
the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 9.2 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 



Fig. 10. Fraction of electrons backscattered from a titanium target as a 
function of the electron beam energy: experimental data (black and grey filled 
symbols) and Geant4 9.3 simulation results with Urban (red empty circles), 
UrbanB (blue empty triangles) and UrbanBRF (brown crosses) multiple 
scattering configurations. 











14 



Fig. 11. Fraction of electrons backscattered from a silicon target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 9.4 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 
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Fig. 13. Fraction of electrons backscattered from a carbon target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.0 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 



Fig. 12. Fraction of electrons backscattered from a germanium target as a 
function of the electron beam energy: experimental data (black and grey filled 
symbols) and Geant4 9.6 simulation results with Urban (red empty circles), 
UrbanB (blue empty triangles) and UrbanBRF (brown crosses) multiple 
scattering configurations. 


implementation related to this parameter. 

Table IX also shows the p-values resulting from the com¬ 
parison of the compatibility with experiment achieved with 
the Urban configuration in Geant4 9.1 and the UrbanB and 
UrbanBRF variants in the same Geant4 version and later. 
The hypothesis of equivalent compatibility with experiment 
is rejected in all cases, with the exception of the UrbanB 
configuration in Geant4 9.1 and 9.2. 

From this analysis one concludes that the multiple scattering 
settings characterizing the UrbanBRF configuration produce 
significantly different compatibility with experiment with re¬ 
spect to the Urban one, which achieves the best compatibil¬ 
ity with backscattering measurements in the earlier Geant4 
versions evaluated in this paper. It is worthwhile to note 
that the multiple scattering settings of the UrbanBRF con¬ 
figuration reproduce those implemented in the recommended 
G4EmStandardPhysics_option3 PhysicsConstmctor, which is 
claimed in in to enable high accuracy simulation. 

A similar categorical analysis estimates the statistical sig¬ 
nificance of the difference in compatibility with experiment 
between simulations based on the Urban configuration in 
Geant4 9.1 and on multiple or single scattering models other 
than the Urban model in the latest Geant4 versions. The results 
are summarized in Table pC| regarding energies above 100 keV. 
In the energy range between 20 and 100 keV the hypothesis of 
equivalence is rejected in all test cases, while below 20 keV 
it is rejected only in test cases involving the Coulomb and 
WentzelBRF configurations. Given the low efficiency of the 
Urban model below 20 keV observed in Table |VII| the test 
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Fig. 14. Fraction of electrons backscattered from a silicon target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.0 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 


Energy (keV) 

Fig. 16. Fraction of electrons backscattered from a tin target as a function of 
the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.0 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 
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Fig. 15. Fraction of electrons backscattered from a copper target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.0 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 


Fig. 17. Fraction of electrons backscattered from a gold target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.0 simulation results with Urban (red empty circles), UrbanB 
(blue empty triangles) and UrbanBRF (brown crosses) multiple scattering 
configurations. 
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Fig. 18. Fraction of electrons backscattered from a magnesium tai'get as a 
function of the electron beam energy: experimental data (black and grey filled 
symbols) and simulation results (empty symbols) with the Urban multiple 
scattering model, complemented by user defined step limitation, in Geant4 
version 9.1 (red circles), 9.2 (green crosses), 9.3 (blue upside down triangles), 
9.4 (magenta squares), 9.6 (turquoise triangles), 10.0 (brown diamonds) and 
10.1 (pink asterisks). 



Fig. 19. Fraction of electrons backscattered from a cadmium target as a 
function of the electron beam energy: experimental data (black and grey filled 
symbols) and simulation results (empty symbols) with the Urban multiple 
scattering model, complemented by user defined step limitation, in Geant4 
version 9.1 (red circles), 9.2 (green crosses), 9.3 (blue upside down triangles), 
9.4 (magenta squares), 9.6 (turquoise triangles), 10.0 (brown diamonds) and 
10.1 (pink asterisks). 


cases where the hypothesis of equivalence is not rejected 
reflect the equivalent incapability of Geant4 multiple scattering 
models to describe backscattering accurately at the lowest 
energies. 


TABLE X 

P- VALUES RESULTING FROM THE COMPARISON OF COMPATIBILITY WITH 
EXPERIMENT BETWEEN SIMULATIONS USING URBAN MULTIPLE 
SCATTERING IN GEANT4 9.1 AND SIMULATIONS USING OTHER 
UNRELATED CONFIGURATIONS IN VERSIONS 9.6, 10.0 AND 10.1, 
CONCERNING ELECTRON ENERGY ABOVE 100 KEV 


Geant4 

Physics 

Fisher 

Pearson 

Barnard 

Boschloo 

Version 

Configuration 

Test 

Test 

Test 

Test 


Coulomb 

1 

1 

1 

1 


GSBRF 

0.026 

0.016 

0.019 

0.19 


WentzelBRF 

1 

1 

1 

1 


WentzelBRFP 

<0.0001 

<0.0001 

<0.0001 

<0.0001 

9.6 

EmLivermore 

<0.0001 


<0.0001 

<0.0001 

EmStd 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


EmOptl 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


EmOpt2 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


EmOpt3 

<0.0001 


<0.0001 

<0.0001 


EmOpt4 

<0.0001 


<0.0001 

<0.0001 


Coulomb 

1 

0.815 

0.889 

1 


GSBRF 

0.026 

0.016 

0.019 

0.19 


WentzelBRF 

1 

0.815 

0.889 

1 


WentzelBRFP 

<0.0001 


<0.0001 

<0.0001 

10.0 

EmLivermore 

<0.0001 


<0.0001 

<0.0001 

EmStd 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


EmOptl 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


EmOpt2 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


EmOpt3 

<0.0001 


<0.0001 

<0.0001 


EmOpt4 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


Coulomb 

<0.0001 


<0.0001 

<0.0001 


CoulombP 

1 

1 

1 

1 


GSBRF 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


WentzelBRF 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


WentzelBRFP 

0.0001 


0.0001 

0.0001 

10.1 

EmLivermore 

0.0001 


0.0001 

0.0001 

EmStd 

<0.0001 


<0.0001 

<0.0001 


EmOptl 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


EmOpt2 

<0.0001 

<0.0001 

<0.0001 

<0.0001 


EmOpt3 

<0.0001 


<0.0001 

<0.0001 


EmOpt4 

<0.0001 


<0.0001 

<0.0001 


FmWVl 

<0.0001 


<0.0001 

<0.0001 


D. Geant4 Coulomb scattering model 

The simulation application instantiates Geant4 Coulomb 
scattering process and its associated model for electron scat¬ 
tering through the respective class constructors and uses them 
in their default configuration. 

A sample of the behaviour of this model with respect to 
experimental data is illustrated in Figs. 20p2 The single 
Coulomb scattering model appears to have evolved from 
Geant4 version 9.1 to 10.0 towards better reproducing ex¬ 
perimental measurements especially at energies below a few 
tens of keV. A degradation of the capability of the model in 
its default configuration is observed in Geant4 10.1. Physical 
performance equivalent to that observed in Geant4 10.0 is 
achieved only by setting the “6* limit” parameter to zero in 
the singleton G4EmParameters class: this simulation config¬ 
uration is identified as CoulombP in this paper. Despite our 
best efforts, we could not retrieve any documentation of the 
semantic change of the construction of Coulomb scattering 
objects in Geant4 10.1, which is responsible for the modified 
behaviour in their default configuration. 
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Fig. 20. Fraction of electrons backscattered from a silicon target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and simulation results (empty symbols) with the Coulomb scattering model 
in Geant4 version 9.1 (red circles), 9.2 (green crosses), 9.3 (blue upside 
down triangles), 9.4 (magenta squares), 9.6 (turquoise triangles), 10.0 (brown 
diamonds) and 10.1 (pink asterisks). 
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Fig. 21. Fraction of electrons backscattered from a germanium target as a 
function of the electron beam energy: experimental data (black and grey filled 
symbols) and simulation results (empty symbols) with the Coulomb scattering 
model in Geant4 version 9.1 (red circles), 9.2 (green crosses), 9.3 (blue upside 
down triangles), 9.4 (magenta squares), 9.6 (turquoise triangles), 10.0 (brown 
diamonds) and 10.1 (pink asterisks). 
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Fig. 22. Fraction of electrons backscattered from a tellurium target as a 
function of the electron beam energy: experimental data (black and grey filled 
symbols) and simulation results (empty symbols) with the Coulomb scattering 
model in Geant4 version 9.1 (red circles), 9.2 (green crosses), 9.3 (blue upside 
down triangles), 9.4 (magenta squares), 9.6 (turquoise triangles), 10.0 (brown 
diamonds) and 10.1 (pink asterisks). 


These qualitative considerations are confirmed by the results 
of McNemar’s test summarized in Table XI which compares 
the compatibility with experiment achieved by the Coulomb 
scattering confiburation in Geant4 10.0 with the outcome of 
goodness of fit tests related to earlier versions. 


TABLE XI 

P-VALUES OF MCNEMAR EXACT TEST COMPARING THE COMPATIBILITY 
WITH EXPERIMENT OF THE COULOMB CONFIGURATION IN GEANT4 10.0 
AND IN OTHER GEANT4 VERSIONS 


Version 

1-20 keV 

20-100 keV 

>100 keV 

9.1 

< 0.001 

< 0.001 

< 0.001 

9.2 

< 0.001 

< 0.001 

< 0.001 

9.3 

< 0.001 

< 0.001 

< 0.001 

9.4 

< 0.001 

0.008 

< 0.001 

9.6 

0.625 

0.754 

0.625 

10.1 

< 0.001 

< 0.001 

< 0.001 


The default Coulomb scattering configuration achieves the 
highest efficiency over all energy ranges in Geant4 versions 
9.6 and 10.0. For convenience, the Coulomb configuration in 
Geant4 10.0 is defined as the reference Coulomb configuration 
for further categorical data analysis. Its compatibility with 
experimental data is compared with the achievements of other 
multiple scattering configurations in Geant4 versions 9.6, 
10.0 and 10.1 in Table XII The hypothesis of equivalent 
compatibility with experiment is rejected for all other models 
implemented in Geant4 10.1 at all energies, as well as for 
the default Coulomb configuration of that version. Equivalent 
behaviour is achieved with non-default settings in Geant4 10.1: 
to the best of our efforts, we could not retrieve mention 
of the major semantic change of the default instantiation 
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of single Coulomb scattering in Geant4 10.1, nor of the 
settings required to restore equivalent functionality to previous 
versions. Regarding earlier Geant4 versions, the hypothesis of 
equivalent compatibility with experiment is not rejected at all 
energies with respect to the WentzelBRF configuration, which 
de facto corresponds to enabling single scattering only in the 
default setting of G4WentzelVIModel. At higher energies the 
Urban configuration achieves equivalent compatibility with 
experiment in Geant4 version 9.6, but not in Geant4 10.0. 
Weak evidence of equivalent compatibility with experiment 


is reported in Table XII regarding the Goudsmit-Saunderson 
multiple scattering model in versions 9.6 and 10.0; further 
details are discussed in section fVI-EI 

The CoulombP configuration in Geant4 10.1 exhibits similar 
behaviour to the reference Coulomb configuration of Geant4 
10.0. Its results are not shown in Figs. [20p2]fo r better clarity 
of the plots, but they are included in Figs. 32|36 Its associated 
efficiency is listed in Table VII McNemar’s test confirms 
its compatibility with the reference Coulomb configuration of 
Geant4 10.0 with 0.01 significance. 

The predefined G4EmStandardPhysics_SS PhysicsCon- 
structor, which instantiates single scattering for various particle 
types, produces results statistically consistent to the CoulombP 
configuration. Since its functionality regarding electron scat¬ 
tering is equivalent to that of the CoulombP configuration, it 
will not be further discussed in the following sections. 


E. Geant4 Goudsmit-Saunderson model 

The implementation of the Goudsmit-Saunderson multiple 
scattering model appears to have evolved from its first release 
in Geant4 9.3 version to the 10.1 version; qualitatively, one 
can observe in Figs. [23p6| that at higher energies the evolution 
of this algorithm has contributed to approach experimental 
backscattering values in simulations based on Geant4 9.6 and 
10.0, followed by an apparent deterioration of compatibility 
in Geant4 10.1. These observations are confirmed by the sta¬ 
tistical analysis of compatibility with experiment summarized 
in Table Ivnl 

The results of the tests reported in Table show that the 
compatibility with experiment achieved with the Goudsmit- 
Saunderson algorithm in Geant4 versions 9.6 and 10.0 is statis¬ 
tically equivalent to that achieved with the Urban configuration 
in Geant4 version. 9.1 in the energy range above 100 keV; 
nevertheless, equivalent compatibility with experiment with 
respect to the Coulomb configuration of Geant4 10.0 in the 
same energy range is assessed only by Fisher test, which is 
notoriously conservative, while more powerful Z-pooled and 
Boschloo tests reject the hypothesis of equivalent compatibil¬ 
ity with experiment with 0.01 significance. Comparisons at 
lower energies fail to establish equivalent compatibility with 
experiment with respect to the 10.0 Coulomb scattering con¬ 
figuration, while they are not physically relevant with respect 
to the 9.1 Urban configuration due to the very low efficiency 


exhibited by both multiple scattering models in Table VII The 
above mentioned comparisons with the Coulomb scattering 
configuration of Geant4 10.0 are documented in Table |Xn] 
The evolution of the Goudsmit-Saunderson algorithm in 
Geant4 10.1 leads to incompatibility with both experimental 


TABLE XII 

P-VALUES RESULTING FROM THE COMPARISON OF COMPATIBILITY WITH 
EXPERIMENT BETWEEN THE SINGLE COULOMB SCATTERING 
CONFIGURATION IN GEANT4 10.0 AND OTHER UNRELATED 
CONFIGURATIONS IN VERSIONS 9.6, 10.0 AND 10.1 


Energy 

Geant4 

Model 

Fischer 

Pearson 

Barnard 

Boschloo 

(kev) 

Version 


Test 

Test 

Test 

Test 



Urban 

< 0.001 


< 0.001 

< 0.001 


9.6 

GSBRF 

< 0.001 


< 0.001 

< 0.001 



WentzelBRF 

0.716 

0.627 

0.683 

0.666 



Urban 

< 0.001 


< 0.001 

< 0.001 

1-20 

10.0 

GSBRF 

< 0.001 


< 0.001 

< 0.001 



WentzelBRF 

1 

0.903 

0.951 

1 



Urban 

< 0.001 


< 0.001 

< 0.001 


10.1 

GSBRF 

< 0.001 


< 0.001 

< 0.001 



WentzelBRF 

< 0.001 


< 0.001 

< 0.001 



Urban 

< 0.001 

< 0.001 

< 0.001 

< 0.001 


9.6 

GSBRF 

< 0.001 


< 0.001 

< 0.001 



WentzelBRF 

0.685 

0.588 

0.682 

0.624 



Urban 

< 0.001 


< 0.001 

< 0.001 

20-100 

10.0 

GSBRF 

< 0.001 


< 0.001 

< 0.001 



WentzelBRF 

0.685 

0.588 

0.682 

0.624 



Urban 

< 0.001 


< 0.001 

< 0.001 


10.1 

GSBRF 

< 0.001 


< 0.001 

< 0.001 



WentzelBRF 

< 0.001 


< 0.001 

< 0.001 



Urban 

0.196 

0.132 

0.144 

0.160 



GSBRF 

0.014 

0.008 

0.009 

0.009 



WentzelBRF 

1 

0.815 

0.889 

1 



WentzelBRFP 

< 0.001 

< 0.001 

< 0.001 

< 0.001 


9.6 

EmLivermore 

< 0.001 


< 0.001 

< 0.001 


EmStd 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



EmOptl 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



EmOpt2 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



EmOpt3 

< 0.001 


< 0.001 

< 0.001 



EmOpt4 

< 0.001 


< 0.001 

< 0.001 



Urban 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



GSBRF 

0.014 

0.008 

0.009 

0.009 



WentzelBRF 

1 

1 

1 

1 



WentzelBRFP 

< 0.001 

< 0.001 

< 0.001 

< 0.001 

>100 

10.0 

EmLivermore 

< 0.001 


< 0.001 

< 0.001 

EmStd 

< 0.001 


< 0.001 

< 0.001 



EmOptl 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



EmOpt2 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



EmOpt3 

< 0.001 


< 0.001 

< 0.001 



EmOpt4 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



Urban 

< 0.001 


< 0.001 

< 0.001 



GSBRE 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



WentzelBRF 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



WentzelBRFP 

< 0.001 

< 0.001 

< 0.001 

< 0.001 


10.1 

EmLivermore 

< 0.001 


< 0.001 

< 0.001 


EmStd 

< 0.001 


< 0.001 

< 0.001 



EmOptl 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



EmOpt2 

< 0.001 

< 0.001 

< 0.001 

< 0.001 



EmOpt3 

< 0.001 


< 0.001 

< 0.001 



EmOpt4 

< 0.001 


< 0.001 

< 0.001 



EmWVI 

< 0.001 

< 0.001 

< 0.001 

< 0.001 


data and the reference Urban and Coulomb scattering config¬ 
urations. 


F. Geant4 WentzelVI Model 

The Geant4 WentzelVI model is used in two configura¬ 
tions evaluated in this paper; WentzelBRF and WentzelBRFP, 
which use it in its default setting and with a polar angle 
threshold, respectively. The latter reflects the setting recom¬ 
mended in the electromagnetic PhysicsConstructors that in¬ 
stantiate the WentzelVI model, such as G4EmStandardPhysics 
and G4EniLivermorePhysics. Their performance in the con¬ 
text of Geant4 10.0 and 10.1 is illustrated in Eigs. [27[ 
and 32p^ respectively. In addition, the predefined 
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Fig. 23. Fraction of electrons backscattered from a magnesium target as 
a function of the electron beam energy: experimental data (black and grey 
filled symbols) and simulation results (empty symbols) with the Goudsmit- 
Saunderson model in Geant4 version 9.3 (blue upside down triangles), 9.4 
(magenta squares), 9.6 (turquoise triangles), 10.0 (brown diamonds) and 10.1 
(pink asterisks). 



Fig. 25. Fraction of electrons backscattered from a gold target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and simulation results (empty symbols) with the Goudsmit-Saunderson model 
in Geant4 version 9.3 (blue upside down triangles), 9.4 (magenta squares), 
9.6 (turquoise triangles), 10.0 (brown diamonds) and 10.1 (pink asterisks).. 
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Fig. 24. Fraction of electrons backscattered from an iron target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and simulation results (empty symbols) with the Goudsmit-Saunderson model 
in Geant4 version 9.3 (blue upside down triangles), 9.4 (magenta squares), 
9.6 (turquoise triangles), 10.0 (brown diamonds) and 10.1 (pink asterisks). 


Energy (keV) 

Fig. 26. Fraction of electrons backscattered from an uranium target as 
a function of the electron beam energy: experimental data (black and grey 
filled symbols) and simulation results (empty symbols) with the Goudsmit- 
Saunderson model in Geant4 version 9.3 (blue upside down triangles), 9.4 
(magenta squares), 9.6 (turquoise triangles), 10.0 (brown diamonds) and 10.1 
(pink asterisks).. 
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G4EmStandardPhysics_WVI PhysicsConstmctor first released 
in Geant4 10.1 configures multiple scattering with the 
WentzelVI model for various particle types, including elec¬ 
trons. 

The WentzelBRF configuration appears to behave similarly 
to Geant4 single Coulomb scattering configuration according 


to the statistical results collected in Tables VII and |XII| up to 
version 10; this similarity also concerns its evolution over the 
Geant4 versions examined in this paper. It achieves its highest 
efficiency in Geant4 10.0. 

Large differences, especially visible at lower energies, are 
observed between the results produced by the WentzelBRF and 
WentzelBRFP configurations in Geant4 versions up to 10.0. 
The two configurations behave similarly in Geant4 10.1. 

The statistical analysis over the earlier Geant4 versions 
documented in Tables |VIH |X| and |XII| confirms that, while 
the WentzelBRF configuration produces statistically equiva¬ 
lent compatibility with experiment with respect to the most 
efficient configurations (Urban in Geant4 9.1 and Coulomb in 
Geant4 10.0), the hypothesis of equivalent compatibility with 
experiment is rejected for the WentzelBRFP one. 

From these results one can infer that in the experimental 
scenarios evaluated in this paper the recommended polar angle 
setting contributes to worsening the reproduction of measured 
backscattering with respect to the default settings. 

The WentzelBRF and WentzelBRFP configurations produce 
statistically equivalent results in Geant4 10.1: McNemar’s test 
fails to reject the hypothesis of compatibility between the 
two categories of goodness-of-fit testing results with 0.01 
significance. 

The behaviour of the predefined G4EmStandard- 
PhysicsWVI PhysicsConstructor is equivalent to that 
of the WentzelBRF and WentzelBRFP configurations in 
Geant4 10.1: this conclusion is assessed by McNemar’s test 
with 0.01 significance. 


G. Geant4 recommended PhysicsConstructors 

As stated in section |IV-B| the evaluation of Geant4 rec¬ 
ommended PhysicsConstructors regarding their simulation of 
electron backscattering concerns Geant4 versions 9.6, 10.0 and 
10 . 1 . 

The fraction of backscattered electrons produced by G4Em- 
StandardPhysics, G4EmStandardPhysics_optionl, G4Em- 
StandardPhysics_option2, G4EmStandardPhysics_option3 
G4EmStandardPhysics_option4 and G4EniLivermorePhysics 
with Geant4 10.0 version is illustrated in Figs. [J7pT| One 
can qualitatively observe that simulation results produced 
by G4EmStandardPhysics, G4EmStandardPhysics_option3, 
G4EmStandardPhysics_option4 and G4EmLivermorePhysics 
fail to reproduce the characteristics of experimental 
distributions over the whole energy range. Sim¬ 
ulations using G4EmStandardPhysics_optionl and 
G4EmStandardPhysics_option2 appear to approach 
experimental data in the higher energy end above 1 MeV. 

These qualitative considerations are reflected in the out¬ 
come of the statistical data analysis. At lower energies the 
recommended PhysicsConstructors appear incapable to repro¬ 
duce experimental data similarly to other Geant4 multiple 



Fig. 27. Fraction of electrons backscattered from a carbon target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.0 simulation results with WentzelBRF (red empty circles) and 
WentzelBRFP (blue crosses) multiple scattering settings and Coulomb single 
scattering model (green empty squares). 
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Fig. 28. Fraction of electrons backscattered from a silicon target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.0 simulation results with WentzelBRF (red empty circles) and 
WentzelBRFP (blue crosses) multiple scattering settings and Coulomb single 
scattering model (green empty squares). 
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Fig. 29. Fraction of electrons backscattered from a copper target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.0 simulation results with WentzelBRF (red empty circles) and 
WentzelBRFP (blue crosses) multiple scattering settings and Coulomb single 
scattering model (green empty squares). 
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Fig. 31. Fraction of electrons backscattered from a gold target as a function 
of the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.0 simulation results with WentzelBRF (red empty circles) and 
WentzelBRFP (blue crosses) multiple scattering settings and Coulomb single 
scattering model (green empty squares). 



Fig. 30. Fraction of electrons backscattered from a tin target as a function of 
the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.0 simulation results withWentzelBRF (red empty circles) and 
WentzelBRFP (blue crosses) multiple scattering settings and Coulomb single 
scattering model (green empty squares). 



Fig. 32. Fraction of electrons backscattered from a carbon target as a 
function of the electron beam energy: experimental data (black and grey filled 
symbols) and Geant4 10.1 simulation results with WentzelBRF (red empty 
circles), WentzelBRFP (blue crosses) multiple scattering settings. Coulomb 
single scattering model in default configuration (green empty squares) and 
with modified pai'ameter settings (pink asterisks). 
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Fig. 33. Fraction of electrons backscattered from a silicon target as a 
function of the electron beam energy: experimental data (black and grey filled 
symbols) and Geant4 10.1 simulation results with WentzelBRF (red empty 
circles), WentzelBRFP (blue crosses) multiple scattering settings. Coulomb 
single scattering model in default configuration (green empty squai'es) and 
with modified pai'ameter settings (pink asterisks). 



Fig. 35. Fraction of electrons backscattered from a tin target as a function of 
the electron beam energy: experimental data (black and grey filled symbols) 
and Geant4 10.1 simulation results WentzelBRF (red empty circles), Wentzel¬ 
BRFP (blue crosses) multiple scattering settings, Coulomb single scattering 
model in default configuration (green empty squares) and with modified 
parameter settings (pink asterisks). 
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Fig. 34. Fraction of electrons backscattered from a copper target as a 
function of the electron beam energy: experimental data (black and grey filled 
symbols) and Geant4 10.1 simulation results with WentzelBRF (red empty 
circles), WentzelBRFP (blue crosses) multiple scattering settings, Coulomb 
single scattering model in default configuration (green empty squares) and 
with modified parameter settings (pink asterisks). 


Energy (keV) 

Fig. 36. Fraction of electrons backscattered from a gold target as a 
function of the electron beam energy: experimental data (black and grey 
filled symbols) and Geant4 10.1 simulation results WentzelBRF (red empty 
circles), WentzelBRFP (blue crosses) multiple scattering settings. Coulomb 
single scattering model in default configuration (green empty squares) and 
with modified pai'ameter settings (pink asterisks). 
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Fig. 37. Fraction of electrons backscattered from a carbon target 
as a function of the electron beam energy: experimental data (black 
and grey filled symbols) and Geant4 10.0 simulation results with 
G4EmLivermorePhysics (red empty circles), G4EmStandardPhysics 
(green crosses), G4EmStandardPhysics_optionl (blue empty upside- 
down triangles), G4EmStandardPhysics_option2 (magenta empty 
squares), G4EmStandardPhysics_option3 (turquoise empty triangles) 
and G4EmStandardPhysics_option4 (brown empty diamonds) 
PhysicsConstructors. 



Fig. 38. Fraction of electrons backscattered from a silicon target 
as a function of the electron beam energy: experimental data (black 
and grey filled symbols) and Geant4 10.1 simulation results with 
G4EmLivermorePhysics (red empty circles), G4EmStandardPhysics 
(green crosses), G4EmStandardPhysics_optionl (blue empty upside- 
down triangles), G4EmStandardPhysics_option2 (magenta empty 
squares), G4EmStandardPhysics_option3 (turquoise empty triangles) 
and G4EmStandardPhysics_option4 (brown empty diamonds) 
PhysicsConstructors. 


scattering configurations. G4EmStandardPhysics_option3 and 
G4EmStandardPhysics_option4, which according to Geant4 
documentation are intended to produce high accuracy sim¬ 
ulations, exhibit negligible efficiency at all energies, as 
well as G4EmLivermorePhysics. At higher energies the ef¬ 
ficiency achieved using G4EmStandardPhysics_optionl and 
G4EmStandardPhysics_option2 is higher than that associated 
with the other recommended PhysicsConstructors in Geant4 
9.6 and 10.0 is approximately a factor two lower than 
that achieved in the most efficient scenarios (Urban mul¬ 
tiple scattering configuration in Geant4 9.1 and Coulomb 
single scattering configuration in Geant4 10.0). The perfor¬ 
mance of G4EmStandardPhysics_optionl is consistent with 
the statement in Geant4 user documentation that this Physic- 
sConstructor is intended for fast, but less accurate simula¬ 
tion. G4EmStandardPhysics exhibits a similar performance in 
Geant4 9.6, while its efficiency drops in Geant4 10.0. 

Eor all the recommended PhysicsConstructors the hypothe¬ 
sis of equivalent compatibility with the most efficient config¬ 
urations (Urban in Geant4 9.1 and Coulomb in Geant4 10.0) 
is rejected with 0.0001 significance above 100 keV. 

As a result of this analysis, one can conclude that in the 
scenario examined in this test significantly better accuracy 
can be achieved with physics configurations other than those 
recommended in Geant4 user documentation. 


H. Computational Performance 


The heterogeneous production environment of the sim¬ 
ulations documented in this paper, consisting of different 
platforms, prevents the absolute comparison of the compu¬ 
tational performance of different physics modeling options. 
Nevertheless, in most test cases the simulations corresponding 
to a group of multiple scattering configurations (Urban model 
and its variants, WentzelVI model variants. Coulomb, GSBRE) 
were executed on the same machine, thus enabling a relative 
comparison of their computational performance at least at a 
qualitative level. 


A sample of results in Figs. and 43 illustrate the 
main features regarding the computational performance of 
backscattering simulations, namely the overhead associated 
with simulating single scattering rather than multiple scatter¬ 
ing, in Geant4 version 9.6. One can observe that the Coulomb 
and WentzelBRF configurations exhibit similar computational 
performance, while the Urban configuration is an order of 
magnitude faster. All recommended PhysicsConstructors are 
substantially faster than the other configurations shown in 
the plots, although at the price of significantly degraded 
compatibility with experiment, as documented in the previous 
sections. 
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Fig. 39. Fraction of electrons backscattered from a copper target 
as a function of the electron beam energy: experimental data (black 
and grey filled symbols) and Geant4 9.6 simulation results with 
G4EmLivermorePhysics (red empty circles), G4EmStandardPhysics 
(green crosses), G4EmStandardPhysics_optionl (blue empty upside- 
down triangles), G4EmStandardPhysics_option2 (magenta empty 
squares), G4EmStandardPhysics_option3 (turquoise empty triangles) 
and G4EmStandardPhysics_option4 (brown empty diamonds) 
PhysicsConstructors. 


VII. Correlation between backscattering and 
ENERGY deposition 

It is physically intuitive that a relation exists between the 
fraction of electrons that are backscattered from a target 
volume and the energy deposited in it. A few examples of 
the energy deposited in the targets used in the simulation 
of electron backscattering are shown in Figs. EHZl they 
illustrate the dependence of the energy deposition on the 
evolution of the Geant4 Urban model from version 9.1 to 10.0 
in Figs. 44p3 and the effects associated with different physics 
configurations in the simulation in Figs. 46p7 Visible differ¬ 
ences are observed both among different Geant4 versions for 
the same physics configuration, and among different physics 
configurations in the same Geant4 version. 

The capability of Geant4 to reproduce experimental mea¬ 
surements of the energy deposited by low energy electrons 
is quantitatively analyzed in Q with respect to two exper¬ 
imental configurations; a longitudinally segmented detector 
and a detector consisting of a bulk volume. The latter closely 
resembles the simulation configuration of the backscattering 
test cases evaluated in this paper. Reference |[T) reports the 
efficiency, representing the compatibility with experimental 
data of energy deposition simulations based on Geant4 ver¬ 
sions 9.1 to 9.6: these values were obtained using the default 


Fig. 40. Fraction of electrons backscattered from a tin target as 
a function of the electron beam energy: experimental data (black 
and grey filled symbols) and Geant4 10.0 simulation results with 
G4EmLivermorePhysics (red empty circles), G4EmStandardPhysics 
(green crosses), G4EmStandardPhysics_optionl (blue empty upside- 
down triangles), G4EmStandardPhysics_option2 (magenta empty 
squares), G4EmStandardPhysics_option3 (turquoise empty triangles) 
and G4EmStandardPhysics_option4 (brown empty diamonds) 
PhysicsConstructors. 


Urban multiple scattering model corresponding to each Geant4 
version and imposing a user-defined step limitation, similar 
to the configuration identified as Urban in this paper. The 
hypothesis was formulated in |[T] that the discrepancies in 
compatibility with experiments obtained with different Geant4 
versions could be attributed to the evolution of the implemen¬ 
tation of the Urban Geant4 multiple scattering algorithm. 

The test of electron backscattering simulation, which is a 
direct effect of multiple scattering algorithms implemented in 
Geant4, allows a quantitative test of this hypothesis: the evalu¬ 
ation whether a correlation exists between the two sets of effi¬ 
ciencies, derived from the validation of energy deposition and 
backscattering simulations, respectively. It is worthwhile to 
note that measures of correlation are not inferential statistical 
tests, but are, instead, descriptive statistical quantities, which 
represent the degree of relationship between two observables. 
Statistical inference concerning the underlying population is 
enabled by the analysis of the significance of the measured 
value through appropriate hypothesis testing methods. 

For the purpose of measuring the correlation between the 
two sets of efficiencies, the data sample over which the 
backscattering efficiency is calculated has been restricted to 
the energy range of the energy deposition test described in 
Q, and the correlation analysis is limited to Geant4 versions 
common to both validation tests. 

Measures of correlation related to these observables are cal¬ 
culated and complemented by inferential statistical tests 1 128), 
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Fig. 41. Fraction of electrons backscattered from a gold target as 
a function of the electron beam energy: experimental data (black 
and grey filled symbols) and Geant4 10.1 simulation results with 
G4EmLivermorePhysics (red empty circles), G4EmStandardPhysics 
(green crosses), G4EmStandardPhysics_optionl (blue empty upside- 
down triangles), G4EmStandardPhysics_option2 (magenta empty 
squares), G4EmStandardPhysics_option3 (turquoise empty triangles) 
and G4EmStandardPhysics_option4 (brown empty diamonds) 
PhysicsConstructors. 


Fig. 43. Relative execution time as a function of beam energy for 
a set of physics configurations simulating electrons backscattered from a 
tungsten target with Geant4 9.6: single Coulomb scattering (black squares), 
WentzelBRF (red circles), G4EmStandardPhysics_option4 (blue triangles) and 
Urban (green diamonds). The results in the plot are scaled with respect to the 
execution time of a simulation with G4EmStandardPhysics. 
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Fig. 42. Relative execution time as a function of beam energy for a set of 
physics configurations simulating electrons backscattered from a carbon target 
with Geant4 9.6: single Coulomb scattering (black squares), WentzelBRF (red 
circles), G4EmStandardPhysics_option4 (blue triangles) and Urban (green 
diamonds). The results in the plot are scaled with respect to the execution 
time of a simulation with G4EmStandardPhysics. 


Fig. 44. Energy deposited in a carbon target as a function of the electron beam 
energy, resulting from simulations with the Urban multiple scattering config¬ 
uration in different Geant4 versions: 9.1 (red circles), 9.2 (green crosses), 9.3 
(blue upside down triangles), 9.4 (magenta squares), 9.6 (turquoise triangles) 
and 10.0 (brown diamonds). 
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Fig. 45. Energy deposited in a molybdenum target as a function of the 
electron beam energy, resulting from simulations with the Urban multiple 
scattering configuration in different Geant4 versions: 9.1 (red circles), 9.2 
(green crosses), 9.3 (blue upside down triangles), 9.4 (magenta squares), 9.6 
(turquoise triangles) and 10.0 (brown diamonds). 



Fig. 47. Energy deposited in a tantalum target as a function of the electron 
beam energy, resulting from simulations with different multiple scattering 
configurations in Geant4 version 9.6: Urban (red circles), UrbanBRF (green 
crosses) WentzelBRF (blue upside down triangles) and Coulomb (magenta 
squares) electron scattering configurations, EmLivermore (turquoise triangles) 
and EmStandard (brown diamonds) pre-packaged PhysicsConstructors. 



c 

o 

o 

CO 


O) 

0) 

c 

CD 

“D 

CD 

H—I 

03 

O 

Q. 

CD 

Q 


1.00 

0.95 

0.90 

0.85 

0.80 

0.75 

0.70 



Cu, Z=29 Geant4 10.0p03 
o Urban v WentzelBRF ^ EmLivermore 

X UrbanBRF □ Coulomb O EmStd 


0.1 0.5 5 50 500 5000 

Energy (keV) 


Fig. 46. Energy deposited in an iron target as a function of the electron 
beam energy, resulting from simulations with different multiple scattering 
configurations in Geant4 version 9.6: Urban (red circles), UrbanBRF (green 
crosses) WentzelBRF (blue upside down triangles) and Coulomb (magenta 
squares) electron scattering configurations, EmLivermore (turquoise triangles) 
and EmStandard (brown diamonds) pre-packaged PhysicsConstructors). 


Fig. 48. Energy deposited in a copper target as a function of the electron 
beam energy, resulting from simulations with different multiple scattering 
configurations in Geant4 version 10.0: Urban (red circles), UrbanBRF (green 
crosses) WentzelBRF (blue upside down triangles) and Coulomb (magenta 
squares) electron scattering configurations, EmLivermore (turquoise triangles) 
and EmStandard (brown diamonds) pre-packaged PhysicsConstructors. 
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Fig. 49. Energy deposited in a lead target as a function of the electron 
beam energy, resulting from simulations with different multiple scattering 
configurations in Geant4 version 10.0: Urban (red circles), UrbanBRF (green 
crosses) WentzelBRF (blue upside down triangles) and Coulomb (magenta 
squares) electron scattering configurations, EmLivermore (turquoise triangles) 
and EmStandard (brown diamonds) pre-packaged PhysicsConstructors. 


11291. The null hypothesis is formulated as the absence of any 


correlation between the two sets of efficiencies, expressed by 
a measure of zero. The alternative hypothesis concerns the 
existence of a positive correlation between the two sets of 
efficiencies associated with Geant4 versions; it corresponds to 
the execution of one-tailed tests. 

Two nonparametric correlation measures, Spearman p cor¬ 
relation 1T30| and Kendall r correlation p31[ , are reported 


in Table XIII along with the associated p-values. Table |XIII| 
also lists Pearson product-moment correlation coefficient |132[ 
and the associated p-value: although more widely known 
in the experimental physics environment, Pearson correlation 
coefficient has a more limited scope, as it describes linear 
correlation only. 

The null hypothesis of no correlation is rejected with 0.01 
significance. Consistent with the alternative hypothesis, one 
can infer that a positive correlation exists. From this analysis 
one can infer that the accuracy of simulation of electron 
backscattering, which is a direct effect of multiple scattering 
modeling, and of simulation of the energy deposited by low 
energy electrons are correlated. 


TABLE XIII 

Correlation between efeiciency at simulating backscattering 

COEFFICIENT AND ENERGY DEPOSITION COMPATIBLE WITH EXPERIMENT 
OVER GEANT4 VERSIONS 9.1 TO 9.6 


Correlation 

Measure 

p-value 

Kendall r 

I 

0.008 

Spearman p 

I 

0.008 

Pearson correlation coefficient 

0.992 

0.0005 


VIII. Conclusion 


This paper has analyzed the capabilities of several Geant4 
multiple scattering models with respect to a large experimental 
data sample, and its evolution over Geant4 versions from 9.1 
to 10.1. In addition, a single scattering algorithm has been 
evaluated. The fraction of electrons backscattered from a semi¬ 
infinite target has proven to be a sensitive probe of multi¬ 
ple scattering algorithms available in Geant4. A significant 
correlation between the accuracy of this observable and the 
accuracy of the simulation of the energy deposition originating 
from low energy electrons has been established on rigorous 
statistical grounds. 

Large variability is observed in the performance of all 
models over the range of Geant4 versions considered in this 
study. The evolution does not always go in the direction 
of improving the compatibility with experiment: statistically 
significant regressions are observed for some Geant4 models 
with respect to their previous behaviour. Although an analysis 
of the software development process in the electromagnetic 
physics domain of Geant4 is outside the scope of this paper, 
the results of this validation test highlight the opportunity 
to strengthen the discipline of change management and the 
traceability of changes, including their side effects. Established 
software process frameworks, such as the Unified Process 
1 133| , CMMI (Capability Maturity Model Integration) |134| 
and the ISO/IEC 15504 Standard | |135| , provide support for 
these and other related disciplines. 

Due to the variability of physics performance affecting all 
models, it is not possible to identify a single Geant4 version 
providing an optimal simulation environment over the whole 
energy range considered in the test. All the evaluated multiple 
scattering models encounter difficulties at reproducing low 
energy backscattering measurements: up to a few tens of keV 
only the single Coulomb scattering model demonstrates some 
capability to describe experimental data realistically, limited 
to Geant4 9.6 and 10.0 versions in its default configuration, 
although at the price of substantially slower computational 
performance. Above a few tens of keV, the Urban model in 
Geant4 9.1, complemented by user defined step limitation, 
demonstrates the best capability to reproduce experimental 
data in a condensed transport scheme. Recommended settings 
advertised for high accuracy, such as UseDistanceToBoundary, 
worsen the compatibility with experiment of this configuration. 

In the investigated scenario the predefined electromagnetic 
PhysicsConstructors do not fulfill the expectations of accuracy 
deriving from their advertisement in Geant4 documentation 
and the statement of their intensive validation in conference 
papers. The recommendations of their use should be revised 
and based on objective grounds, documenting the scope to 
which they are pertinent and quantifying the capabilities of the 
embedded physics configurations with respect to experimental 
data. 

The quantitative assessment documented in this paper, along 
with the results of the validation test documented in |(T), 
allow Geant4 users to optimize the physics configuration of 
simulation applications involving electrons of energy up to a 
few tens of MeV, either as primary particles or as secondary 
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interaction products. 
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